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1. Macdonald polynomials 



Macdonald polynomials P\(x;q,t) are orthogonal symmetric polynomials which are 
the natural multivariable generalisation of the continuous q-ultraspherical polyno- 
mials C n (x; /3\q) which, in their turn, constitute an important class of hyper- 
geometric orthogonal polynomials in one variable. Polynomials C n (x; /3\q) can be 
obtained from the general Askey-Wilson polynomials || through a specification of 
their four parameters (see, for instance, @), so that C n (x; (3\q) depend only on one 
parameter /3, apart from the degree n and the basic parameter q. In an analogous 
way, the Macdonald polynomials P\(x; q, t) with one parameter t could be obtained 
as a limiting case of the 5-parameter Koornwinder's multivariable generalisation of 



kfi the Askey-Wilson polynomials [TO |. 

The main reference for the Macdonald polynomials is the book [17[, Ch. VI, 
where they are called symmetric functions with two parameters. Let K = Q(q, t) 
>- ! be the field of rational functions in two indeterminants q, t; K[x] = K[si, . . . , x n ] be 

the ring of polynomials in n variables x = (x±, . . . , x n ) with coefficients in K; and 
Kfx] 1 ^ be the subring of all symmetric polynomials. The Macdonald polynomials 
P\{x) = P\(x;q,t) are symmetric polynomials labelled by the sequences A = {0 < 
Ai < A2 < . . . < A n } of integers (dominant weights). They form a K basis of 
and are uniquely characterised as joint eigenvectors of the commuting g-difference 
operators 

H k P x = h k]X P x , k=l,...,n, (1.1) 
normalised by the condition 

P\= k aa' mx, «aa = 1 » ( K AA' e K) (1.2) 
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where, for each /i, m M (x) stands for the monomial symmetric function: m M (x) = 
Xi 1 ■ ■ ■ % V n with the sum taken over all distinct permutations v of /i, and -< is the 
dominance order on the dominant weights 



n n 

X'^X {|A'| = |A|; E^<E A i> k 

j=k j=k 



n 



The commuting operators if; have the form 

H * = E II v ik I ( II T i>*i I ' i = l,...,n, (1.3) 



JC{l,...,n} 



\~j\=i ' V fce{i,...,n}\J 

where 




i _ i 

«i*= — |g|<l,|t|<l. (1.4) 



The T^. stands for the g-shift operator in the variable Xf (T q ^ x f)(xi. 



i X r 



f(xi, . . . , gxj, . . . ,x n ). The operators (|1.3| ) were introduced for the first time in 



PTf and are the integrals of motion for the quantum Ruijsenaars model, which is 
a relativistic (or q-) analog of the trigonometric Calogero-Moser-Sutherland model. 
They are called sometimes Macdonald operators in the mathematical literature. 

are 



The corresponding eigenvalues hk-\ in (|Lj 



/%•••/%) H = 1 H 2 • ( L5 ) 

h<-<5k 

The polynomials P\ are orthogonal 

-i- / / ^P x (x;q,t)Py(x-q,t) A(x) = 0, A ^ A' (1.6) 

(2m) n J x\ J x n 

|a;i|=l |x n |=l 

with respect to the weight 



A(»i,-,^) = n ( t^r - (L7) 

For instance, for n = 3, 

m oo = 1, ^ooi = xi + x 2 + x 3 , m u = XXX2 + XXX3 + X2X3, m 00 2 = xj + xl + xl, 

mm = Xix 2 x 3 , m i2 = X\x\ + x\x 2 + Xixl + x\x 3 + x 2 x\ + 
"iii2 = x\x2X 3 + xix\x 3 + xiX2x'l 1 m 22 = x\x\ + x\x\ + x\x\, m 03 = x\ + x\ + x\. 

-PoOO = m 000) -PoOl = m 001) -Poll = ^-011) -Po02 = ^002 + l-tf~ 9 ^ m 011; 

-r ill — Willi, -MD12 — m 012 H fZ^p Willi) 

PU2 = TO112, P 22 = m 22 + (1 ~ff (? 1 f +<?) WI112, 
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^003 - ^003 H 1^27 ^oi2 H (i_,t)(i_,2t) — m m- 

Rodrigues type of formula for Macdonald polynomials was recently found in 



IT, p~2|, [UJ . In the limit q f 1 Macdonald polynomials turn into JacA; polynomials 
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A multivariable function/polynomial can be called special function if it is some 
recognised (classical) special function in the case of one variable and if it is common 
eigenfunction of a complete set of commuting linear partial differential/ difference 
operators defining a quantum integrable system. Macdonald polynomials are special 
functions in the above-mentioned sense. They diagonalise the integrals of motion 
Hj of the quantum Ruijsenaars system. 

For any special function in many variables one can set up a general problem 
of its factorisation in terms of one-variable (special) functions. For the Macdonald 
polynomials P\ this would mean finding a factorising integral operator M such that 

n 

M:P A (x;g,t)^n $ A,,(%). 

3=1 

For some particular special functions (or, in other words, for some particular 
quantum integrable systems) such an operator might simplify to a local transform 
which is a simple change of variables, from x to y. This happens for example in the 



case of ellipsoidal harmonics in M™ (see and |23j for n = 3), correspondingly, in 



the case of quantum Neumann system, where the transform x 1— > y is the change of 
variables, from Cartesian to ellipsoidal. 

As it is shown further on, in Section 3, the factorising operator for the (sym- 
metric) Macdonald polynomials has to be non-local, i.e. to be some linear integral 
operator. Moreover, it is explicitly described in the first two non-trivial cases (when 
n = 2 and n = 3) in terms of the Askey- Wilson operator. 

The formulated factorisation problem also makes sense in the limit to the Li- 
ouville integrable systems in classical mechanics, becoming there the well-known 
problem of separation of variables (SoV) in the Hamilton- Jacobi equation. 

The kernel of the factorising integral operator M turns in this limit into the 
generating function of the separating canonical transformation. 



2. Hypergeometric polynomials f\ and (p\ in one 
variable 

In this Section we introduce the hypergeometric polynomials fx/ (fx of one variable 
each constituting a basis which is conjugated to Jack/Macdonald polynomials with 
respect to the factorising integral transform M. First of all, we describe the pro- 
cedure of lowering the order of (basic) hypergeometric functions which will lead us 
to these new sets of interesting polynomials in one variable. We will use (becoming 
already) standard notations of |§ for the (basic) hypergeometric series and other 
formulas of the g-analysis. 
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In 1927 Fox pi found an interesting relation between hypergeometric functions: 



p 

p r q 



ai + mi, GS2, • • • , a p 



h, ... ,b 



5 u q 



E 

j=0 



j\ (6l) f -(6 9 )i p q 



a 1 +j,...,a p +j 
h+j,...,b q +j 



(2-1) 



When ai = &i ( |2~T|) gives the expansion of p F q 
a 



in terms of functions of the 



type p _iFg„! 



. When mi is a positive integer, the series on the right of (|2.1| 



terminates, and we have (for the case of a\ = b\ and q = p — 1) the following relation 



F 1 



&i + mi, a 2 



, U/^, . . . , <Alp _ 



6i,6 2 . . . ,6 p _i 



(2.2) 



m i 



E (-y)J(-mi) j (a2)j-(a P )j 77. 
j\ (6i)j-(6p-i)j P-i^P-a 



3=0 



2 + j, • • ■ , Op + j 

6 2 + j, . . . , Vi + i 



Supposing that a 2 = 6 2 + m 2 , a 3 = 6 3 + m 3 , . . . , a p _i = 6 p _i + m p _i with some 
non- negative integers m^, = 2, . . . ,p — 1, we then iterate the relation (|2.2j ) further 
on and, finally, using the binomial theorem, 



y 



conclude that the function 

fm 1 ,...,m p ^ 1 = (1 — y) P j pFp-i 



is a polynomial in y of the cumulative degree J27=i m j- 



(i-yr a 



hi + mi, b 2 + m 2 , . . . , 6 p _i + m P -i, a P . 

6i,62...,6p-i ' y . 



As for a g-analog of the reduction formula ( |2.2| ) we refer to (1.9.4) in (see 
also 0). It was proved for the first time in in a more general case (although in 
different notations than ones in || which are adopted here). Following ||, consider 
the g-analog of the Vandermonde formula in the form 



2^1 



Q i Q 



bp-i 



(bp-i;q) n 



(2.3) 



where m and n are non-negative integers such that m > n, then we can reduce the 
order of the basic hypergeometric function through the following equalities (\y\ < 1): 



ai, . . . , ctp-i, b p -iq r ' 



(2.4) 



E (ai,...,fl P -i;q)n n (g n ,q m ;q) k mn+k 

(q,bi,—,b P -T,g)n " (q,b P -i;q)k " 



n=0 

oo m 



fc=0 

(ai,...,a p _i;g)„(g~ m ;g) fc 



EE (6i,...,b p _2;g)n(g;g)„_fc(g,6p-i;g)fe 
n=0 fc=0 

E (g- m ,ai,...,a p -i;g) fc / - J „m\A;_-( 2 J J. 



_-j\fc^mn+fc— nfe+( 2 ) 



A:=0 



aig fc , . . . ,a p _ig fc 
b iq k ,...,bp_ 2 q k ' q > yq 



m—k 
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Then again, iterating it and using the g-binomial theorem, 



Wo 



q,y 



(2/;?)« 



\y\ < 1 



\q\ < 1 



we conclude that, for non-negative integers m k , k — 1, . . . ,p — 1, the function 

(y;q)oo , 



( Pmi,...,m p -i 



(ya p q m ^-+ m ^;q) c 



b iq mi ,...,b p _ iq m ^,a p 

Ox , . . . , 0, 



1 u p-l 



is a polynomial in y of the cumulative degree S"=i "V 

Now we can connect integers rrij to a partition A = {Ai, . . . , A n }, namely: 

m j = Vu (> 0) , j = 1, . . . , n - 1 . 

Put also (g E E) 

6j = Ai-Aj+i + l-jp, a j = b j + m j , j = 1, . . . , n - 1 , a n = A x - A n + 1 - n# . 
Let us define the following function 



f x (y) :=y Al (l-y) 1 - f "„F„_i 



h + rnt,..., b n _i + m n _i, a n 



&1> • • • > 

Obviously, this function is a polynomial in y of the form 

Am 



(2.5) 



fc=Ai 



The polynomials /x(y) were introduced in [13[] and they are factorised polynomials 
of one variable for the multivariable Jack polynomials. In |22|, the corresponding 
sine-kernel (of Gegenbauer type) factorising the A2 Jack polynomials was described 
in detail. In the sequel we concentrate on the g-analogs of these polynomials, the 
(fix(y), which were introduced in fTl|. 

Put again rrij = A J+1 — A.,-, j — 1, . . . , n — 1, but 

= g Ai-A i+1 +l-i 3 ? r/; /, // »' . J = l,...,n-1, a n = gAx-An+l-ng 

We will also use the parameter t which is connected to g in the following way 

t := q 9 . 

Then the polynomials (f \ are defined as follows: 

b 1 q m \...,b n -x(f nn -\ 



<p\(y) '■= y 1 (y;q)i-n 9 n <Pn- 



61, . . . ,b n . 



q,y 



■2.6) 



and, like f\(y), expand in y as (|2.5|). 
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In [H| we have found several useful representations for these polynomials in 
addition to their definition (12.61). Let us list here some of them. Introduce notations: 



|A| = EV 



First of all, the coefficients Xk in (|2.5|) have the following explicit representation: 

-un\ A i- fc (q~ 1 t n ;<i)k-x 1 



Xk 



q Xl k ,a 1 ,...,a n 
? Al - fc+2 t- n ,6i,...,6 n _i 



(2.7) 



It is easy to give simpler expressions for some of Xk such as 
X\i = !, Xx n = t 



» \l — |A| 'j-J (Pj ^Xj-XijV', gOAn-An- 

Let us give few first polynomials in the case n = 3 

¥?ooo = 1, y%)i = 1 + y, <fou = l + ^y, 

i I (l+q)(l-t) 1-qt 2 



(2. 



,„ _ 1 , l+qt+t-qt 2 -t 2 -qt B , 1 2 

y?oi2 - i h i2(i_ gt 2 ) — y + jsv. 



m _ 1 , (l+g)(l-f 2 ) , (l+t)(l-gf 2 ) 2 



_ -■ | (l+g+g 2 )(l-f) , (l+g+g 2 )(l-t) 2 , . , . 

¥^003 — J- -T t {l-q 2 t 2 ) >J ^ t 2 {l+qt)(l-qt 2 ) * ^ t 3 (l+t)(l+qt)(l-qt 2 ) & 

There is also a simple formula for f\(t n ) 

j=i k 1 > <i)x ]+1 -x 1 



l-q 2 t 



(2.9) 



There is a nice representation of the polynomials (f \ in terms of the <$£> g-Lauricella 
function § H | 



<P\{y) = V 



Ai 



>D 



v\ 6!,... A-i 

qt~ n q Xln y 



i 9) a i; • • • ) n— 1 



with 



a' = q f- n q 



n Ai-A„_ 



j = 1, ... ,72 - 1 



(2.10) 



The 0^1 g-Lauricella function of n — 1 variables z^ is a multivariable generalisation 
of the basic hypergeometric series and is defined by the formula 



>D 



a'- h' h' 
a ) u ii ■ ■ ■ i u n-l 



q; z%, . . . , z n _\ 



g (a'i q)k 1+ ...+k n ^ j-j- 1 (b'j ; q) kj Zj 



ki 



,...,fc n _ 1= o ( c 5 9)fei+-+fe»-i j=i 
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using which we get the most explicit representation of our polynomials ip\ 



>n-l > 1 



My) =y Xl ( m<i Xl ~ Xn - j+1+ltj ~ n ->v)^ + i-^ 



x 



x E ■•■ E (9*" n 9 Aln+fcl+ - +fen - 1 2/;?)A„ 1 - fel -...- fen _ 1 (2/;g) fcl+ ... + fe_ 1 

fci=0 fc n _i=0 

x n- J — ■ (2-ii) 

Finally, these polynomials satisfy the following g-difference equation 

n 

£ (_i)* r^* (1 _ gfct-fcy) ( y; g ), (g^H-^; </)„_* K_ k , x Vx {q k y) = (2.12) 

k=0 



where hk-\ are given by ( |1.5| ) and we assume /io ; A = 1- 

Let us give few remarks about this new class of basic hypergeometric polynomials 
in one variable. 

Remark 1. First of all, we stress that our way of extracting polynomials from 
hypergeometric series is quite different from the usual one. Indeed, all classical 
orthogonal g-polynomials of one variable are obtained by just terminating the corre- 
sponding hypergeometric series. For instance, the generic Askey- Wilson orthogonal 
g-polynomials which appear on the 4^3 level are defined as follows 



p n {x; a, b, c, d\q) = const ■ 4 ( 



q n 1 abcdq n 1 ,ae* y ,ae 



ab, ac, ad ' ^ 



x = cos 9 . 



In contrast, to extract the polynomials ip\ at the level n n _i we use the procedure 
of order reduction ( |2.4| ) of the basic hypergeometic functions with the specific choice 
of upper and lower parameters, namely: when the ratio of an upper parameter and 
one of the lower parameters is equal to q mi where rrii are non-negative integers. The 
procedure of order reduction is thus an important second possibility (in addition to 
simple termination) in order to get polynomials out of hypergeometric series. 

Remark 2. The importance of polynomials (f\ stems from the fact that they are the 
factorised polynomials of one variable for the multivariable Macdonald polynomials. 
Notice that the polynomials <p\, as well as Macdonald polynomials P\, are labelled 
by the dominant weights. The multivariable polynomials $a combined from the 
one- variable polynomials ipy. 

n-l 

®\(yi, ■■■,Vn)-= y^n II fxiVk) , 

k=l 

satisfy the following multiparameter spectral problem (cf. fl2.12j )) 

$a(3/i, • • • , 2/n-i, qy n ) = K-,x $\(yi, ■■■,Vn), (2-13) 
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(-l) fc r^ k (1 - qH-%) (y f , q) k {q k+1 r n y 3] q) n . k /i n _ fe;A x 

fe=o 

x$ x (y 1 ,...,q k y j ,...,y n ) = 0, j = l,...,n- 1, 

with t/ie same set of spectral parameters (hi-\, . . . , h n .\) ( |1.5| ) as in the spectral 
problem (|1 . 1|) for the Macdonald polynomials Pa- Hence, one can introduce the 
commuting operators H{ , i = 1, . . . ,n defined by their eigenfunctions &\{y) and 
eigenvalues h^\- Since the spectrum (/ii ; a, • • • , h n .\) coincides with that ( |1.5|) of the 
Macdonald polynomials, the two sets of commuting operators are isomorphic and 
there has to exist an intertwining linear operator M 

M H\ x) = H\ v) M (Hl x) = Hi) . 

Actually, to any choice of the normalisation coefficients c\ in 

M : P x (x) ^ c A $ A (y) 

there corresponds some intertwiner M. The problem is to select a factorising oper- 
ator M having an explicit description in terms of its integral kernel or its matrix in 
some basis in K[x] w . In |L4|, [T~5|1 we have found such an operator as well as its in- 
version in the first two non-trivial cases, when n = 2 and n = 3. It appears that the 
factorising operator M can be expressed in those cases through the Askey- Wilson 
operator. 



3. The cases n = 2 and n = 3 



Let us first describe the integral operator performing the separation of variables 
in the Ai Macdonald polynomials (we skip the trivial case of the purely coordinate 
SoV x\ t 2 — > x± = (xix^ 1 ) 1 / 2 ). Our main technical tool is the famous Askey-Wilson 
integral identity [[| § 

1 / dx (x 2 ,x~ 2 ;q) ao _ 2{abcd;q) oc i \ 

2iri J x (ax,ax~ 1 ,bx,bx~ 1 ,cx,cx~ 1 ,dx,dx~ 1 ;<j)oo {q^b^ac^dficfid^cd-^ao ' V " / 

^ abed 

The cycle T abcd depends on complex parameters a, b, c, d and is defined as follows. 
Let C z ^ r be the counter-clockwise oriented circle with the center z and radius r. If 
max(|a| , |6| , |c| , \d\ , \q\) < 1 then T abcd = C 0i i. The identity ( p.l|) can be continued 
analytically for the values of parameters a, b, c, d outside the unit circle provided 
the cycle T abcd is deformed appropriately. In general case 

Fabcd = C*0,1 + X] (^zq k ,e ~ C z -l q -k )E ) , 

z=a,b,c,d fc>0 
\z\q k >l 

e being small enough for C z ±i q ±k >E to encircle only one pole of the denominator. 
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Put 

— 7 -1 — £ 1 — 1 — 

a = yq 2 , b = y q 2 , c = rq 2 } a = r q 2 . 

We will use the notation for the contour obtained from T abcd by these substitu- 
tions. Introduce also the following useful notation 

C q (u; x, y) := (uxy, vxy" 1 , ux^y, ux~ l y~ l ] q)^ . (3.2) 

Now let 

oi = (3 = g, y = y~, x = x_, r = t~ 1 y + , x± = (x l x 2 tl ) 2 ~ , y± = (yiy^ 1 ) 2 ' ■ 
Introduce the kernel A4(y + , yJ\xJ): 

M(y+ y-\x_) = ( x ~ tilo (xl,xz 2 ; g)oo C q (t; r V) 

Assuming £ to be an arbitrary complex parameter, we introduce the integral operator 
acting on functions f(x\,X2) by the formula 

(MJ)( yi ,y 2 ) = -L / — A% + ,y_|z) f{r*£y + z, rhv+z' 1 ) ■ (3-4) 



Theorem 1 ( |T5| ) The operator ( \3.4 ),( \3^a ) performs the factorisation of (or, 
in other words, the SoV for) the A\ Macdonald polynomials: 

M c : P x (x u x 2 ) -> c A ,£ 9?a(i/i) ^(2/2) , (3.5) 

where <p\(y) is the factorised (or separated) polynomial and the normalisation coef- 
ficient c\£ is equal to 

CA? = t -2A 1+ A 2 £|A| (t|g)A al (3 6) 



Note that the relation (|3.5|) is equivalent [15[ to the product formula for the 
continuous g-ultraspherical polynomials [^0], §]. 

The kernel of the inverse operator Mr 1 has the form 

~ , , (1 -?)(?; g)L (z/-,z/- 2 ; g)oo 2;-, Hf- 1 ^ 
yvi(x + , — 



37 j 



2B q (-g,2g) £,(H;y_,a;_) C q {t;y-,H^ 
with the following substitutions for the contour T: 

a = -9, (3 = 2g, x = V-, V = x-, r = t~%£~ 1 
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Theorem 2 (||15||) The inversion of the operator ( $-4) is given by the formula 

dz 



{M^f){x u x 2 ) 



1 

2tH 



M(x+,x„\z) f(t*£ 1 x + z,t*£ l x+z 



n t _1 / 2 5- 1 i + ,i_ 

-9,2s 



(3-7) 

The operator 1 provides an integral representation for the A\ Macdonald polyno- 
mials in terms of the factorised polynomials <Pxi,x 2 {y) 



Me 1 : c A ,5 <p\{yi) <f\(y2) -> P\(x 1 ,x 2 ) . 



(3.8) 



In contrast to the formula ( |3.5|) which paraphrases an already known result, the 
formula ( |3.8| ) leads to a new integral relation for the g-ultraspherical polynomials. 
Note that for positive integer g the operator M^ 1 becomes a g-difference operator 
of the order g (cf. |nj and [jT5[]). 

Now we describe the factorising operator M and its inversion in the case of A 2 
Macdonald polynomials. Introduce the following operator M acting on functions 
f(x%, x 2 , x 3 ) by the formula 



(Mf)( yi ,y 2 ,y 3 ) 



1 

2tH 



dx. 



M((yty 2 )^, (2/1/2/2) 2 \x-) x 



s,2 9 



V+,V- 



with the kernel 



M(y+,y- I x-) 



;i - ?)(?; <?)L(£-, 9)00 ; y-> 

2B q (g,2g) C q (t*;y-,x-.) C q (t; x-.,y+t~* 



and the following substitutions: a = g, (3 = 2g, r = t ?y + , y = y_, x = x. 



[XiX 2 



y± = (2/i2/ 2 t1 ) 2 - 



Theorem 3 ( |T4| ) The operator M transforms any A 2 Macdonald polynomial P\ 
into the product 

M : P x { Xl ,x 2 ,x 3 ;q,t) -> c A ^+^+a 3 
of factorised polynomials (pxiX 2 x 3 (y) of one variable 



¥>\{y) = y Xl (y,q)i-3g sfo 



-2 n l-X 3l t -l q \-X 2 i lliV 



t~ z q l 



The normalisation coefficient cx equals 

f A 2 -4Ai 

{t 2 ;q)x :n {t 2 ;q)x 32 {t]q) 

A 21 



c x = r 



(t 3 ;q)\ 31 (t;q)x 32 (t 2 ;q)x 21 
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Theorem 4 (||14||) The inverse operator M 1 is the integral operator 

1 r dy. 



(M 1 f)(xi,x 2 ,x 3 ) 



2ni 



V- 



M 



(X\X 2 )z / X\\2 



X3 \X 2 



y- x 



-9.3s 



... t^(xix 2 )^y- t2(x 1 x 2 ) 2 
x / , — — , x 3 



•'V. 



X3V- 



with the kernel 



M(x + ,x_ | y_) 



(1 - qf^jy 2 ., y_ 2 ; q)oo C q (t; s_, z+) 
2B q (-g, 3g) £ g (t _ 3; y_, ar_) a;+ 



(3.9) 



It provides a new integral representation for the A 2 Macdonald polynomials in terms 
of the factorised polynomials ( fx 1 ,\ 2 ,\ 3 {y) of one variable 



(3.10) 



For positive integer g the operator M 1 iwns into the q-difference operator of order 
9' 



M- 1 : f(yi,y 2 ,y3)^Yl &l 

fc=i 

where £fc(r, zs gwen fry 

, fe(fe-i) r 

e*(r,y) = (-l)V^- 2 



(XlX 2 )2 /Xl\2 



/(g 9+fe -,g 29 - fc -,x 3 ) (3.ii) 

^3 X 3 



g-2k..-2\ (try,tr l y;g) k {try 1 ,tr 1 y 1 ;g) g - k 



(t 2 ;q) g (q k y 2 \q) 



9 + 1 



Remark 3. As was found in p4| , [15| the operators Mg and M are closely related 
to a slightly more general integral operator M a p which, in turn, is closely related to 
the fractional g-integration operator I a . 

Remark 4. The apparent similarity of formulas for the operator M in cases n = 2 
and n = 3 is not a coincidence. Its explanation relies on the reduction gl(2) C gl{3), 
see [fL5 . 
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